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ABSTRACT 

Historically  NASA  has  trained  teams  of  astronauts  hy  bringing  them  to  the  Johnson  Space  Center  in  Houston  to 
undergo  generic  training,  followed  by  mission-specific  training.  This  latter  training  begins  after  a crew  has  been 
selected  for  a mission  (perhaps  two  years  before  the  launch  of  that  mission). 

While  some  Space  Shuttle  flights  have  included  an  astronaut  from  a foreign  country,  the  International  Space  Station 
will  be  consistently  crewed  by  teams  comprised  of  astronauts  from  two  or  more  of  the  partner  nations.  The  cost  of 
training  these  international  teams  continues  to  grow  in  both  monetary  and  personal  terms.  Thus,  NASA  has  been 
seeking  alternative  training  approaches  for  the  International  Space  Station  program. 

Since  1994  we  have  been  developing,  testing,  and  refining  shared  virtual  environments  for  astronaut  team  training, 
including  the  use  of  virtual  environments  for  use  while  in  or  in  transit  to  the  task  location.  In  parallel  with  this  effort, 
we  have  also  been  preparing  applications  for  training  teams  of  military  personnel  engaged  in  peacekeeping  missions. 
This  paper  will  describe  the  applications  developed  to  date,  some  of  the  technological  challenges  that  have  been 
overcome  in  their  development,  and  the  research  performed  to  guide  the  development  and  to  measure  the  efficacy  of 
these  shared  environments  as  training  tools. 

INTRODUCTION 

Many  Space  Shuttle  flights  include  an  astronaut  from  a foreign  country.  These  astronauts  usually  carry  out  most  of 
their  training  at  the  Johnson  Space  Center  in  Houston  or  in  special  facilities  at  other  NASA  centers.  These  astronauts 
often  relocate  their  entire  family  to  Houston.  Thus,  the  support  of  international  crews  for  the  Space  Shuttle  has  been  at 
great  cost  to  the  nation  from  which  the  astronaut  comes  and  has  had  a high  personal  cost  for  both  the  astronaut  and  his 
or  her  family.  The  International  Space  Station  (ISS)  program  is  a partnership  between  NASA,  the  European  Space 
Agency,  the  Japanese  equivalent  (NASDA),  the  Canadian  Space  Agency,  and  the  Russian  Space  Agency.  The  four- 
person  crews  that  will  staff  the  ISS  will  be  derived  from  the  partner  nations  and  will  also  include  guest  astronauts  from 
other  countries.  For  years  those  responsible  for  training  astronauts  for  the  ISS  have  grappled  with  the  issue  of  training 
these  international  crews.  In  order  to  reduce  the  costs  of  such  training  and  to  address  the  “political”  issues  of  where 
training  should  occur,  the  NASA  training  community  has  sought  means  of  training  teams  of  astronauts,  to  some  extent, 
while  each  astronaut  remains  in  his  or  her  home  country. 

Since  1994,  we  have  been  exploring  the  application  of  shared  virtual  environments  for  the  training  of  international 
teams  of  astronauts.  This  task  has  encompassed  a number  of  issues.  Among  these  are  necessary  communications 
bandwidth,  communications  latencies,  data  transfer  protocols,  interaction  metaphors,  human  figure  representations, 
necessary  fidelity  (visual,  audio,  haptic),  navigation  methods,  training  transfer,  and  the  psychology  of  team  building. 

In  addition,  the  long  duration  and  complexity  of  ISS  missions  precludes  training  personnel  for  all  possible  operations 
prior  to  the  mission’s  commencement.  Moreover,  certain  skills  and  knowledge  acquired  prior  to  a mission  may 
degrade  during  the  month’s-long  duration  of  typical  ISS  stays.  Thus,  VEs  are  being  investigated  to  serve  as  delivery 
mechanisms  for  “just-in-time”  training.  In  this  case  these  systems  could  provide  training  just  prior  to  the  performance 
of  a critical  task  for  which  no  training  was  delivered  prior  to  the  flight  or  for  which  skills  or  know'ledge  may  have 
degraded.  It  should  also  be  possible  to  transmit  new  or  updated  databases  to  such  space-based  VEs  to  address  specific 
unforeseen  problems. 

HUBBLE  SPACE  TELESCOPE  MISSION 

In  December,  1993  the  Space  Shuttle  captured  the  Hubble  Space  Telescope  (HST),  and  astronauts  carried  out  a 
complex  set  of  procedures  to  correct  its  mirror’s  optical  problems,  replace  and  upgrade  certain  instruments,  and  carry 
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out  planned  maintenance  (see  Figure  1).  This  mission,  by 
most  measures,  was  the  most  challenging  mission  ever 
performed  in  the  history  of  the  United  States’  manned  space 
program.  Preparation  for  the  mission  was  the  most  careful 
and  detailed  and  brought  unique  resources  to  bear  for  the 
training  of  both  astronauts  and  the  ground-based  flight  team. 
In  order  to  achieve  the  maximum  state  of  readiness,  virtual 
environments  were  used,  for  the  first  time,  in  the  training  of 
the  flight  team  [Loftin,  1995],  This  use  of  virtual 
environments  for  training  joins  a small  set  of  applications 
that  demonstrate  the  efficacy  of  this  technology  for  training. 
Shortly  after  the  conclusion  of  this  mission,  interest  in  using 
shared  virtual  environments  to  train  international  teams  of 
astronauts  emerged  as  a high  priority  within  NASA’s 
training  community.  NASA,  the  University  of  Houston,  and 
the  Fraunhofer  Institute  for  Computer  Graphics  joined 
together  to  pursue  an  initial  effort  to  determine  the 
feasibility  of  a shared  virtual  environment  for  team  training. 
The  context  for  the  feasibility  study  took  the  form  of  a 
simple  extravehicular  (EVA)  simulation  that  engaged  two 
astronauts,  one  located  at  JSC  and  the  other  in  Darmstadt, 
Germany  to  reenact  a portion  of  the  1993  HST  mission.  In 
this  simulation,  the  two  astronauts  changed  out  the  Solar 
Array  Drive  Electronics  (SADE)  in  the  HST. 

Months  of  preparation  culminated  in  a demonstration  on 
September  20,  1995,  as  Astronaut  Bernard  Harris 
(physically  located  in  Houston  at  the  Johnson  Space  Center)  entered  a virtual  environment  with  Astronaut  Ulf  Merbold 
(physically  located  at  the  Fraunhofer  Institute  for  Computer  Graphics  in  Darmstadt,  Germany).  Their  shared 
environment  consisted  of  models  of  the  Space  Shuttle  payload  bay  and  the  Hubble  Space  Telescope.  The  two 
astronauts  spent  over  thirty  minutes  performing  the  major  activities  associated  with  the  changeout  of  SADE.  Their 
work  included  the  real-time  hand-off  of  the  replacement  SADE  in  exchange  for  the  original  SADE.  At  the  conclusion 
of  the  task  the  two  astronauts  shook  hands  and  waved  good-bye  [Loftin,  1997],  Interviews  with  the  astronauts  after 
this  experiment  revealed  their  support  of  and  interest  in  this  mode  of  training.  They  confirmed  its  utility  for  mission 
planning  and  the  familiarization  phase  of  training  and  acknowledged  its  value  in  reducing  travel  and  personal 
segregation  from  their  families  and  home  environments.  The  results  of  this  singular  experiment  have  led  to  sustained 
funding  from  NASA  to  continue  developing  the  technologies  needed  for  this  type  of  training  and  to  design  and 
conduct  experiments  to  identify  human-human  and  human-machine  issues  that  require  solution  if  this  approach  to 
training  is  to  become  part  of  the  baseline  training  for  the  International  Space  Station. 

INTERNATIONAL  SPACE  STATION 

Based  on  the  success  of  the  Houston-Germany  experiment  described  in  the  previous  section,  we  have  been  developing 
a testbed  around  the  International  Space  Station.  The  training  focus  of  this  testbed  is  the  conduct  of  scientific 
experiments  and  the  execution  of  repair  and  maintenance  operations  within  the  1SS.  Models  of  the  interiors  of  selected 
ISS  modules  have  been  created  (see  Figure  2)  and  populated  with  both  highly  detailed,  interactive  models  (polygonal) 
of  the  elements  for  which  training  has  been  developed  along  with  detailed  but  non-interactive  models  (texture  mapped) 
of  the  remaining  interior  elements.  The  racks  of  equipment  for  which  training  is  delivered  in  this  virtual  environment 
comprise  the  Biotechnology  Facility  (BTF).  Prototypes  of  these  racks  are  currently  on-board  the  Russian  Mir  Space 
Station,  affording  opportunities  to  investigate  training  transfer  for  currently-performed  tasks  (see  Figure  3). 

A major  distinction  between  this  experimental  testbed  and  the  environment  used  in  the  Houston-Germany  experiment 
described  earlier  is  the  transfer  of  the  training  context  from  extravehicular  activities  (EVA)  to  intravehicular  activities 
(IVA).  Whereas  in  the  EVA  environment  astronauts  were  represented  as  “suited”  individuals,  employing  rather 
simple  graphics  to  represent  their  bodies,  in  the  IVA  environment,  high  fidelity  human  models  were  developed  to 
serve  as  avatars  for  the  actual  participants  in  virtual  environment  training.  This  approach  sets  a much  higher  standard 
for  the  fidelity  with  which  the  humans  are  represented  and  challenges  the  developers  to  find  a reasonable  compromise 
between  fidelity  and  graphics  display  performance  (see  Figure  4).  Training  participants  are  instrumented  with  sensors 


Figure  1.  A team  of  astronauts  prepares  to 
maintain  the  Hubble  Space  Telescope  during  the 
December,  1993  mission 
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Figure  2.  Cross-sectional  view  of  ISS  science  module 


Figure  4.  Avatars  are  used  to  represent  astronauts  in 
the  ISS  virtual  environment 


that  enable  real-time  tracking  of  critical  portions  of  their  bodies  (typically  the  head,  torso,  and  wrists).  Using  fast 
inverse  kinematics  [Tolani,  1996],  we  have  implemented  avatar  motion  for  the  arms  that  reasonably  represents  the 
actual  motion  of  the  human  arm  during  the  operations  carried  out  by  the  tracked  real  arm.  Work  is  underway  to 
further  refine  these  inverse  kinematics  and  achieve 
even  faster  performance  and  more  natural  positioning  of 
the  entire  arm  [Yang,  1997].  This  work  seeks  to 
combine  the  best  features  of  the  analytic  approach 
described  by  Tolani  and  Badler  [Tolani,  1996]  with 
experimental  results  from  neurophysiological  studies 
[Lacquaniti,  1982;  Soechting,  1989a;  Soechting, 

1989b].  Leg  motion  is  not  implemented  in  the  current 
environment  since  it  does  not  play  a major  role  (at  least 
in  the  sense  of  walking  on  the  Earth’s  surface)  for 
location  in  microgravity.  Eventually,  we  will  have  to 
tackle  this  problem  since  leg  position  is  important  in 
some  tasks  where  the  human  must  exert  substantial 
forces  or  torques  on  objects. 

The  current  testbed  has  been  implemented  at  three  sites 
to  support  experiments  in  training  two  astronauts  while 
other  personnel  (trainers  or  managers)  are  able  to  view 
the  actions  of  the  trainees  via  a “stealth”  mode.  That  is, 
the  stealth  view  is  a stereo  camera  view  that  can  be  positioned  anywhere  in  the  virtual  environment  hut  has  no 
manifestation  in  that  environment  that  can  be  perceived  by  the  trainees.  Final  experimental  procedures  are  now  being 
developed  in  preparation  for  controlled  experiments  using  this  testbed.  The  goal  of  these  experiments  will  be  a 
training  transfer  study  to  the  “real”  hardware  (either  in  the  Russian  Mir  Space  Station  or  on  the  ground  once  the 
equipment  is  returned  to  Houston). 


'Mm 

Figure  3.  Model  of  the  Biotechnology  Facility  in 
the  ISS  and  on  the  Russian  Mir  Space  Station 
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It  should  be  noted  that  a major  current  thrust  is  directed  at  enabling  the  use  of  virtual  environments  in  space  to  support 
“just-in-time”  training.  Such  training,  in  low  Earth  orbit,  might  include  environments  shared  by  astronauts  in  the  ISS 
and  ground-based  engineers  and  other  professionals.  Such  an  approach  has  been  suggested  for  both  emergency  repair 
of  complex  systems  and  the  conduct  of  medical  procedures  in  response  to  serious  injuries  or  illnesses  that  might  be 
encountered  during  long-duration  space  missions. 

PEACEKEEPING  OPERATIONS 

With  support  from  the  Office  of  Naval  Research,  a group  (comprised  of  the  University  of  Houston,  the  University  of 
Pennsylvania,  George  Mason  University,  LinCom  Corporation,  and  Lockheed-Martin)  has  been  conducting  basic 
research  and  applying  its  results  to  develop  a prototype  application  to  address  training  of  military  personnel  in 
Peacekeeping  Operations.  Work  has  been  carried  out  in  two  areas  of  basic  research:  team  training  in  virtual 
environments  and  the  utility  of  virtual  environments  to  support  nonverbal  human-human  communication. 

The  context  for  the  virtual  environment  application  that  is  under  development  is  a checkpoint  in  a Bosnian  setting  (see 
Figure  5).  This  scenario  offers  a versatile  testbed  for  investigating  the  use  of  shared  virtual  environments  for  training 
small  units  (fire  teams  and  squads)  for  carrying  out  peacekeeping  operations  in  a potentially  hostile  location  where  the 
focus  is  on  employing  the  Rules  of  Engagement  in  the  face  of  potentially  unfriendly  civilians  and  an  enemy  operating 
in  a covert  manner. 


Figure  5 - A U.S.  soldier  manning  a Bosnian 

This  application  is  being  used  to  investigate  the  degree  of  training  transfer  to  similar,  real-world  settings  (see  Figure 
6).  In  particular,  we  seek  to  challenge  the  trainees  with  distracters  and  to  enable  them  to  experience  the  consequences 
of  failure  to  employ  correct  procedures  and  follow  the  Rules  of  Engagement.  A typical  training  scenario  might 
employ  a distracter  (an  individual  running  across  the  soldier’s  field  of  view)  aimed  at  drawing  the  attention  of  the 
covering  soldier  away  from  his  teammate  while  that  partner  is  engaged  in  checking  the  identification  of  the  driver  of  a 
car  stopped  at  a checkpoint.  If  the  trainee’s  attention  is  drawn  from  “covering”  his  teammate,  the  consequence  could 
be  the  driver  taking  a weapon  from  concealment  and  shooting  the  soldier  who  is  attempting  to  check  his  identification. 
Thus,  the  results  of  distraction  from  one’s  assigned  mission,  can  be  forcefully  brought  home  to  the  trainee. 

In  tandem  with  this  application  development  we  have  been  conducting  a series  of  experiments  in  both  team  training 
transfer  [Bliss,  1997]  and  nonverbal  communication.  The  first  experiments  have  focused  on  team  navigation  skills.  A 
larger  study  is  now  underway,  in  collaboration  with  the  Army  Research  Institute  (Orlando  Field  Unit),  to  determine 
how  well  navigation  skills  acquired  as  a team  in  a virtual  environment  transfer  to  the  setting  represented  by  the  virtual 
environment. 
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Figure  6.  Scene  from  Peacekeeping  Operations 
Environment 


The  second  set  of  studies  has  not  yet  been  subjected  to  peer  review,  but  results  of  two  separate  experiments  have  been 
analyzed.  These  results  demonstrate  that  subjects  in  typical  virtual  environments  can  reliably  recognize  stereotyped 
and  some  subtle  human  facial  expressions.  The  reliability  of  recognition  is  comparable  to  that  achieved  by  the 
subjects  viewing  photographs  depicting  the  same  expressions.  Experiments  are  now  being  designed  to  investigate  the 
degree  to  which  the  application  that  we  have  developed  can  provide  training  that  is  transferable  to  real-world 
analogues. 

IN  SITU  TRAINING 

NASA  astronauts  have  consistently  noted  the  need  for  training  during  spaceflight,  even  relatively  short-duration  Space 
Shuttle  missions.  Apparently  they  believe  that  some  skills,  such  as  landing  the  Space  Shuttle,  deteriorate  during  a two- 
to-three-week-long  mission.  In  response,  NASA  has  developed  a PC-based  system,  delivered  via  a laptop 
computer,  for  landing  refresher  training.  As  mission  lengths  grow  (e.g.,  International  Space  Station  stays  and 
future  Lunar/Mars  missions)  from  weeks  to  months  and  years,  NASA  must  systematically  address  in  situ  training  for 
crews.  Clearly,  much  of  this  training  can  be  provided  via  standard  computer-based  techniques.  However,  there  are  a 
significant  number  of  tasks  that  demand  interactive,  three-dimensional  representations  in  order  to  achieve  the  needed 
degree  of  realism  (both  visually  and  through  body  orientation  and  manipulation  by  hand).  For  example,  astronauts 
currently  plan  their  body  placements  for  many  tasks  using  virtual  environments.  Thus,  research  and  development  is 
underway  to  address  the  delivery  of  some  training  in  space  using  virtual  environment  technology.  The  first  virtual 
environment  system  will  be  flown  aboard  the  Space  Shuttle  in  mid- 1998  as  a part  of  Neurolab  (STS-90).  Although 
this  system  is  intended  for  use  in  a human  perception  experiment,  its  development  and  flight  clears  the  way  for  future 
space-based  experiments  that  utilize  virtual  environments  for  training.  It  is  anticipated  that  such  an  experiment  will  be 
conducted  during  a mission  in  late  1999  or  early  2000.  The  success  of  this  type  of  experiment  will  likely  lead  to  the 
incorporation  of  a virtual  environment  within  the  International  Space  Station. 

In  addition  to  applications  during  long-duration  Space  Shuttle  missions  and  aboard  the  International  Space  Station 
(both  of  which  occupy  low  earth  orbits),  it  is  probable  that  virtual  environments  will  be  required  for  any  human 
mission  to  the  lunar  surface  or  to  Mars.  In  the  latter  case,  transit  times  will  exceed  one  year  with  total  mission  time 
occupying  three  or  more  years.  Moreover,  communication  lags  can  reach  thirty  minutes  and  there  can  be  significant 
periods  (months  in  length)  with  no  communication  at  all  due  to  particular  Earth-Sun-Mars  alignments. 

IMPLICATIONS  FOR  NATO 

Other  papers  in  this  workshop  have  addressed  the  use  of  virtual  environments  for  a variety  of  military  training 
applications.  The  specific  projects  described  in  this  paper,  while  not  designed  for  typical  warfighting  training,  do 
provide  a basis  for  developing  and/or  improving  virtual  environment  training  for  a variety  of  modern  military 
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missions.  In  particular,  the  application  of  virtual  environments  for  peacekeeping  mission  training  should  be  of  interest 
to  NATO,  given  its  current  role  in  Bosnia. 

The  use  of  virtual  environments  (both  individual  and  shared)  for  small  unit  mission  planning  and  training  offers  a 
number  of  benefits.  For  example,  unit  commanders  can  plan  and  review  tactics  with  appropriate  superiors  and 
subordinates.  Whole  units  (or  parts  thereof)  can  rehearse  entire  missions  or  critical  mission  elements  to  resolve  such 
issues  as  timing  and  personnel  placement.  Perhaps  most  importantly  many  options  for  attaining  a specific  mission 
objective  can  be  quickly  explored  during  the  mission  planning  phase.  Such  an  approach  will  likely  result  in  a 
significant  increase  in  the  probability  of  mission  success  and  a concomitant  reduction  in  the  likelihood  of  casualties 
(both  military  and  civilian). 

Since  NATO  is  a multi-national  organization  joint  operations,  even  at  the  small  unit  level,  are  likely.  Shared 
environments  offer  the  ability  for  units,  comprised  of  personnel  from  more  than  one  service  and/or  more  than  one 
nation,  to  rehearse  a mission  while  they  are  still  in  their  “home”  locations  or  even  in  transit  to  the  site  of  operations. 
Such  an  application  of  virtual  environments  could  reduce  response  time.  In  addition,  the  time  spent  in  these  shared 
environments  could  lead  to  increased  levels  of  trust  and  cohesion  among  individuals  who  have  never  met  prior  to  the 
conduct  of  a mission.  Finally,  a characteristic  of  modern  military  operations  is  the  fluidity  of  team  composition. 
Teams  may  be  constituted  for  a specific  objective  within  the  context  of  a larger  mission.  Such  teams  may  only  exist 
for  hours  or  days,  at  most.  The  achievement  of  mutual  trust  and  team  cohesion  may  be  impossible  if  traditional  team 
training  approaches  are  used.  Again,  shared  virtual  environments  between  participants  in  their  “home”  locations  or 
while  in  transit  to  the  site  of  operations  may  have  give  these  fluid  teams  the  “edge”  needed  to  succeed  in  the  modern 
battlespace. 
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